Multi-codec camera system and image acquisition program

ABSTRACT

A multi-codec camera system includes a multi-codec camera that acquires video in the form of multiple compression-rate image data and a determination module that selects image data from the multiple compression-rate image data acquired by the multi-codec camera according to a predetermined reference, wherein when a predetermined object is detected from the video acquired by the multi-codec camera, the determination module selects image data at a compression rate according to the size of the detected object.

CROSS REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese PatentApplication JP 2005-348641 filed in the Japanese Patent Office on Dec.2, 2005, the entire contents of which being incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to a multi-codec camera system and an imageacquisition program that use a multi-codec camera that acquires video inthe form of multiple compression-rate image data so as to provideoptimal image data according to various conditions to a client.

2. Description of the Related Art

In a network camera system that acquires video via a network in relatedart, a camera connected to the network selects the codec of acquiredimage data (for example, MPEG: Moving Picture Experts Group or JPEG:Joint Photographic Experts Group) and the selected codec image data istransferred to a server or a client (see JP-A-2003-204518,JP-A-2004-146959 and JP-A-2005-136613, for example).

On the other hand, in recent years, a multi-codec camera that supportsmultiple types of codec is available. By using such a camera, it ispossible to build a system that handles multiple codec image data.

SUMMARY OF THE INVENTION

However, in a system using a multi-codec camera, although same videoimages can be simultaneously acquired using multiple types of codec, nosystem can effectively use these images. That is, no system can handleimage data optimal to delivery according to various conditions, such asthe acquired video and the network load, so that no system sufficientlytakes advantage of the convenience of video acquisition by a multi-codeccamera.

According to an embodiment of the invention, there is provided amulti-codec camera system including a multi-codec camera that acquiresvideo in the form of multiple compression-rate image data and adetermination module that selects image data from the multiplecompression-rate image data acquired by the multi-codec camera accordingto a predetermined reference. When a predetermined object is detectedfrom the video acquired by the multi-codec camera, the determinationmodule selects image data at a compression rate according to the size ofthe detected object.

According to the embodiment of the invention, since a predeterminedobject is detected from the video acquired by the multi-codec camera andimage data at a compression rate according to the size of the detectedobject is selected, image data at an optimal compression rate for thesize of the detected object can be transferred to a server and a client.

When the detected object is larger than a predetermined size, image dataat a first compression rate (MPEG, for example) is selected, while whenthe detected object is smaller than the predetermined size, image dataat a second compression rate (JPEG, for example), which is lower thanthe first compression rate, is selected. That is, a lower compressionrate is selected for smaller detected object that demands higherquality, while a higher compression rate is selected for larger detectedobject that does not demand high quality, so as to give a high priorityto the transfer speed.

According to another embodiment of the invention, there is provided amulti-codec camera system including a multi-codec camera that acquiresvideo in the form of multiple compression-rate image data, and a serverthat is connected to the multi-codec camera via a network and has adetermination module that selects image data from the multiplecompression-rate image data acquired by the multi-codec camera accordingto a predetermined reference. The determination module of the serverselects image data at a compression rate according to the condition ofthe network load from the multiple compression-rate image data acquiredby the multi-codec camera.

According to the embodiment of the invention, since image data at anoptimal compression rate is selected for the condition of the networkload, image data at a compression rate optimal to the successivelyvarying conditions of the network load can be transferred to a serverand a client.

When the condition of the network load is higher than a predeterminedreference value, the determination module selects image data at a firstcompression rate (MPEG, for example), while when the condition of thenetwork load is lower than the predetermined reference value, thedetermination module selects image data at a second compression rate(JPEG, for example) that is lower than the first compression rate. Thatis, image data at a higher compression rate is used for a higher networkload, allowing a reduced network load.

According to another embodiment of the invention, there is provided amulti-codec camera system including a multi-codec camera that acquiresvideo in the form of multiple compression-rate image data, a server thatis connected to the multi-codec camera via a network and has adetermination module that selects image data from the multiplecompression-rate image data acquired by the multi-codec camera accordingto a predetermined reference, and a client connected to the network. Theserver saves the multiple compression-rate image data acquired by themulti-codec camera, and the client uses image data at a firstcompression rate when the client searches the image data saved in theserver for image data that the client wants to acquire, while the clientacquires image data at a second compression rate that is higher than thefirst compression rate, when the client acquires the searched image datafrom the server.

According to the embodiment of the invention, since multiplecompression-rate image data are simultaneously saved in the server, whenthe client searches image data saved in the server, the client canquickly perform the searching, for example, by using image data at thelower compression rate captured as still images, while when the clientacquires the searched image data from the server, the client canefficiently transfer image data at the higher compression rate capturedas video images.

According to another embodiment of the invention, there is provided animage acquisition program executed on a multi-codec camera systemincluding a multi-codec camera that acquires video in the form ofmultiple compression-rate image data and a determination module thatselects image data from the multiple compression-rate image dataacquired by the multi-codec camera according to a predeterminedreference. As program processes in the determination module, the imageacquisition program includes the steps of detecting a predeterminedobject from the video acquired by the multi-codec camera and selectingimage data at a compression rate according to the size of the detectedobject.

According to the embodiment of the invention, since a predeterminedobject is detected from the video acquired by the multi-codec camera andimage data at a compression rate according to the size of the detectedobject is selected, image data at an optimal compression rate for thesize of the detected object can be transferred to a server and a client.

According to another embodiment of the invention, there is provided animage acquisition program executed on a multi-codec camera systemincluding a multi-codec camera that acquires video in the form ofmultiple compression-rate image data, and a server that is connected tothe multi-codec camera via a network and has a determination module thatselects image data from the multiple compression-rate image dataacquired by the multi-codec camera according to a predeterminedreference. As a program process in the determination module of theserver, the image acquisition program includes the step of selectingimage data at a compression rate according to the condition of thenetwork load from the multiple compression-rate image data acquired bythe multi-codec camera.

According to the embodiment of the invention, since image data at anoptimal compression rate is selected for the condition of the networkload, image data at a compression rate optimal to the successivelyvarying conditions of the network load can be transferred to a serverand a client.

According to another embodiment of the invention, there is provided animage acquisition program executed on a multi-codec camera systemincluding a multi-codec camera that acquires video in the form ofmultiple compression-rate image data, a server that is connected to themulti-codec camera via a network and has a determination module thatselects image data from the multiple compression-rate image dataacquired by the multi-codec camera according to a predeterminedreference, and a client connected to the network. The image acquisitionprogram includes the steps of, as a program process in the server,saving the multiple compression-rate image data acquired by themulti-codec camera, and, as a program process in the client, using imagedata at a first compression rate when the client searches the image datasaved in the server for image data that the client wants to acquire,while acquiring image data at a second compression rate that is lowerthan the first compression rate, when the client acquires the searchedimage data from the server.

According to the embodiment of the invention, since multiplecompression-rate image data are simultaneously saved in the server, whenthe client searches image data saved in the server, the client canquickly perform the searching, for example, by using image data at thelower compression rate captured as still images, while when the clientacquires the searched image data from the server, the client canefficiently transfer image data at the higher compression rate capturedas video images.

Therefore, the invention provides the following advantage. That is, inthe system using the multi-codec camera, the system selects an optimalcodec according to the condition, so that optimal image data inaccordance with the network load and requests from the user can bequickly provided.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a first schematic configuration diagram of the multi-codeccamera system according to this embodiment;

FIG. 2 is a second schematic configuration diagram of the multi-codeccamera system according to this embodiment;

FIG. 3 is a configuration diagram showing modules necessary for themulti-codec camera system according to this embodiment;

FIG. 4 shows an exemplary GUI; and

FIG. 5 is a software module correlation diagram showing the relationshipamong the modules.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the invention will be described below with reference tothe drawings.

Base Environment for this Embodiment

FIG. 1 is a first schematic configuration diagram of the multi-codeccamera system according to this embodiment, and FIG. 2 is a secondschematic configuration diagram of the multi-codec camera systemaccording to this embodiment. That is, as shown in FIG. 1, a baseenvironment for this embodiment is a system capable of managing one ormore multi-codec cameras 10, which includes a multi-codec camera 10, aserver 20 that acquires, analyzes and saves video from the multi-codeccamera 10, a client 30 that remotely controls the server 20 and viewsthe video, and a network N that connects the above components. Theclient 30 and the server 20 are equipped with functions for acquiringvideo from the multi-codec camera 10, switching the type of codec of thecaptured image or remotely controlling simultaneous acquisition via thenetwork.

The multi-codec camera 10 herein is an image input apparatus capable ofcapturing image data processed at multiple types of compression rates.This embodiment employs an image input apparatus capable of capturing,for example, both high compression-rate MPEG-4 (Moving Picture ExpertsGroup phase 4) image data and low compression-rate JPEG image data.

The client 30 is equipped with a function that changes the setting ofthe server 20, a function that acquires video accumulated in the server20 and a video analysis result, and a function that switches to a modein which video is directly acquired from the multi-codec camera 10.

As shown FIG. 2, this embodiment can also be applied to a systemincluding a management client 30 a having both a server function and aclient function, and a plurality of multi-codec cameras 10. Thisembodiment can also be applied to a system in which each multi-codeccamera 10 has a determination module that normally belongs to a serverand operates without involving any network.

System Necessary for this Embodiment

FIG. 3 is a configuration diagram showing modules necessary for themulti-codec camera system according to this embodiment. That is, theserver 20 is equipped with a “client delivery module 21” that deliversvideo and video analysis results as well as video that the server 20 hasto the client 30, a “server load analysis module 22” that checks thecurrent load on the server 20, a “network condition analysis module 23”that checks the current condition of the network, a “video saving module24” that saves video acquired from the multi-codec cameras 10 based on apreset rule, a “video acquisition rule determination module 25” thatdetermines a video acquisition rule from the condition of the server 20,the condition of the network and the type of an alert, a “videoacquisition module 26” that can simultaneously acquire multiple-codecvideo from the multi-codec camera 10, a “video analysis module 27” thatcan detect an object in motion and an object not in motion from thevideo, an “alert reception module 28” that can receive an alert from themulti-codec camera 10 or the “video analysis module 27”, a “videoanalysis result DB (database) 29 a” that saves video analysis results,and a “video DB 29 b” that saves video.

The client 30 is equipped with a “client reception module 31” that canreceive video and video analysis results from the server 20 as well asthe condition of the server 20, a “video acquisition mode switchingmodule 32” that determines how to acquire video based on the receivedresult, a “video acquisition module 33” that can acquire video from themulti-codec camera 10, a “search screen module 34” that displays asearch screen, a “reproduction module 35” that reproduces accumulatedimages in the server 20 and a “client display module 36” that allows theclient 30 to display imaging results.

The priority in the multi-codec setting can be manually set and FIG. 4shows an exemplary GUI (Graphical User Interface) for performing thesetting. In this exemplary GUI, from the top to bottom, it is possibleto set which codec (for example, MPEG4 or JPEG) is given priority andwhich codec is used to save video, and set the frame rate, resolution,compression level and color level of the respective types of codec. Inthis embodiment, these parameters are automatically set according to thecondition.

The above modules are implemented as program processes executed on theserver 20 and the client 30, each of which includes a computer. Specificexamples and roles of the modules applied in this embodiment will bedescribed below.

[Video Acquisition Module]

The video acquisition module acquires video from the multi-codec camera10.

[Client Delivery Module]

The client delivery module delivers video and video analysis results,the condition of the server 20, and instructions to the client 30.

[Server Load Analysis Module]

The server load analysis module analyzes the conditions of the CPU(Central Processing Unit) and disk access of the server 20.

[Network Condition Analysis Module]

The network condition analysis module analyzes the condition of thenetwork load based on, for example, the planned frame rate at whichinformation will be received from the multi-codec camera 10 and theactually receivable frame rate, as well as the response time (Ping) ofcommunication with the multi-codec camera 10 and the client 30.

[Video Analysis Module]

The video analysis module detects an object in motion or an object notin motion from video and analyzes, for example, the size of the detectedobject.

[Video Saving Module]

The video saving module saves video from each of the multi-codec cameras10 in the database using a pre-specified codec.

[Alert Reception Module]

The alert reception module receives an alert from the multi-codec camera10 or the video analysis module to notify the contents of the alert tothe “video acquisition rule determination module.”

[Video Acquisition Rule Determination Module]

The video acquisition rule determination module determines what codecshould currently be used to acquire video from the multi-codec camera 10and what codec should be used to deliver the video to the client 3 basedon the load on the server 20, the communication condition of the networkand the alert information.

[Video Analysis Result DB]

The video analysis result DB is a database that saves video analysisresults. The data saved in the video analysis result DB is retrievedwhen the client 30 reproduces the saved video.

[Video DB]

The video DB is a database that saves video. The data saved in the videoDB is retrieved when the client 30 reproduces the saved video.

[Client Reception Module]

The client reception module receives video, video analysis results,instructions from the server 20 and the condition of the server 20.

[Video Acquisition Mode Switching Module]

The video acquisition mode switching module switches the videoacquisition mode of the client 30 to directly acquire video for displayfrom the multi-codec camera 10 depending on the instruction from theserver 20.

[Search Screen Module]

The search screen module acquires information necessary for displaying asearch screen on the client 30 from the server 20.

[Reproduction Module]

The reproduction module acquires information necessary for displayingvideo accumulated in the server 20 on the client 30 from the server 20.

[Client Display Module]

The client display module displays video or video analysis resultsacquired from the server 20 or the multi-codec camera 10.

FIG. 5 is a software module correlation diagram showing the relationshipamong the modules. The operation of the system will be described withreference to the correlation diagram.

<Description of the Operation>

(1) The operation of switching the types of codec when an object inmotion with a certain size (preset reference size) or greater (a closerobject in motion) is detected, when an object in motion with a certainsize or smaller (a farther object in motion) is detected, or when alertinformation is received from the camera. The flow of the operation willbe described in (1-1) to (1-6) below.

(1-1) The “video acquisition module 26” acquires video in apre-specified high-priority codec or in multiple types of codec from themulti-codec camera.

(1-2) The “video analysis module 27” detects an object in motion or anobject not in motion based on the video.

(1-3) The “video analysis module 27” notifies the size of the object inmotion or the size of the object not in motion to the “alert receptionmodule 28.” When an alert is received from the multi-codec camera 10,the “alert reception module 28” acquires the contents of the alert (suchas detection of an object in motion, detection of an object not inmotion and detection of sound).

(1-4) The “alert reception module 28” notifies the contents of the alertto the “video acquisition rule determination module 25.”

(1-5) The “video acquisition rule determination module 25” sends aninstruction to change how to acquire video to the “video acquisitionmodule 26” according to the contents of the notification.

When an object in motion with a size greater than a certain value (or anobject not in motion) is detected, or when sound detection is receivedfrom the multi-codec camera 10, the image acquisition method is switchedto the codec by which MPEG video is delivered to the client, while whenan object in motion (or an object not in motion) with a size smallerthan a certain value is detected, the image acquisition method isswitched to the codec by which JPEG video is delivered to the client.The image acquisition method can also be switched to the codec by whichthe video is saved in the [video DB 29 b] of the server. These settingscan also be registered in advance.

(1-6) The “video acquisition module 26” changes the codec of the videoacquired from the multi-codec camera 10 as necessary.

(2) The operation of switching the types of codec when the condition ofthe network load or the condition of the server load becomes worse. Theflow of the operation will be described in (2-1) to (2-7) below.

(2-1) The “video acquisition module 26” acquires video in apre-specified high-priority codec or in multiple types of codec from themulti-codec camera.

(2-2) The “network condition analysis module 23” determines thecondition of the network load based on the frame rate setting, thecurrently receivable frame rate and the result of a network load checkcommand. When the network load exceeds a predetermined threshold value,the condition of the network is notified to the “video acquisition ruledetermination module 25.”

(2-3) The “video acquisition rule determination module 25” sends aninstruction to change how to acquire video to the “video acquisitionmodule 26” according to the contents of the notification from the“network condition analysis module 23.”

When there is no (or very small) communication load on the network, theimage acquisition method is switched to receive both JPEG and MPEGvideo, while when the communication load on the network is lower than acertain threshold value and an alert notification currently does notrequire audio, the image acquisition method is switched to receive JPEGvideo (MPEG video when audio is required). When the frame rate of JPEGimage acquisition is lower than the set value or when the communicationload on the network is higher than the certain threshold value, theimage acquisition method is switched to receive MPEG video. When thecommunication load is even higher and exceeds another certain thresholdvalue, the image acquisition method is switched to receive H264 codecvideo from the multi-codec camera 10. These settings can also beregistered in advance.

(2-4) When the “server load analysis module 22” has determined that theserver load is high, the “server load analysis module 22” sends a codecacquisition instruction for display to the client via the “clientdelivery module 21.”

(2-5) When the “client reception module 31” of the client receives thecodec acquisition instruction sent from the “client delivery module 21”of the server, the “client reception module 31” notifies it to the“video acquisition mode switching module 32”, and the “video acquisitionmodule 33” contained in the client starts acquiring display videodirectly from the multi-codec camera 10 (see the broken-line arrow inFIG. 5).

(2-6) The “client display module 36” merges the video with the videoanalysis result acquired from the “client reception module 31” anddisplays the video to the user.

(2-7) When the network load decreases, the “network condition analysismodule 23” notifies the improved network condition to the “videoacquisition rule determination module 25” and the “video acquisitionrule determination module 25” switches the video acquisition methodagain.

(3) The operation when the server saves JPEG images for searchingpurposes and the client uses the JPEG images for searching. The flow ofthe operation will be described in (3-1) to (3-4) below.

(3-1) The “video acquisition module 26” acquires both MPEG and JPEGcodec video from the multi-codec camera 10.

(3-2) The “video saving module 24” saves JPEG images regularly orwhenever an object in motion is detected as thumbnail images used whenthe client performs searching. On the other hand, MPEG video may bealways recorded.

(3-3) When the client sends a request for displaying the search screen,the “search screen module 34” acquires the thumbnail images from theserver and the “client display module 36” displays the search screen.

(3-4) When the client selects a thumbnail image, the “reproductionmodule 35” acquires MPEG video from the server based on the imaging timerecorded together with the thumbnail image and the “client displaymodule 36” starts reproducing the MPEG video.

According to the embodiment described above, detection of an object inmotion in the video, for example, detection of an object in motionwithin a predetermined distance from the camera or detection of soundcan trigger alarm notification to automatically switch to the MPEG codecfor transmission of audio and images. When the object in motion isoutside the predetermined distance, the codec is switched to JPEG forhigher resolution, which does not allow acquiring sound butcorrespondingly allows transmitting sharper video for each frame than inthe MPEG codec.

Since both JPEG and MPEG codec video is saved in the server, byconsidering the network load and the network configuration, when theacquisition frame rate of JPEG images that the server acquires becomeslower than a certain value, higher priority can be given to the MPEG orH264 codec video while reducing the JPEG frame rate, allowingtransmitting more seamless video.

In a system that manages a plurality of network cameras, when the serverload becomes high and exceeds a certain threshold value, the server canacquire and record only MPEG video and the client can start JPEG videoacquisition. In this way, the user can acquire higher quality videoaccording to various conditions than currently achievable in a systemacquiring video via a network. Also, by using a lower frame-rate JPEGimage as a search key when captured images are reproduced, a searchsystem for accumulated MPEG video can be easily built.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. A multi-codec camera system comprising: a multi-codec camera thatacquires video in the form of multiple compression-rate image data; anda determination module that selects image data from the multiplecompression-rate image data acquired by the multi-codec camera accordingto a predetermined reference, wherein when a predetermined object isdetected from the video acquired by the multi-codec camera, thedetermination module selects image data at a compression rate accordingto the size of the detected object.
 2. The multi-codec camera systemaccording to claim 1, wherein when a predetermined object is detectedfrom the video acquired by the multi-codec camera and the detectedobject is larger than a predetermined size, the determination moduleselects image data at a first compression rate, while when the detectedobject is smaller than the predetermined size, the determination moduleselects image data at a second compression rate that is lower than thefirst compression rate.
 3. The multi-codec camera system according toclaim 1, wherein when a predetermined object is detected from the videoacquired by the multi-codec camera and the detected object is largerthan a predetermined size, the determination module selects to acquireimage data at a first compression rate together with audio data, whilewhen the detected object is smaller than the predetermined size, thedetermination module selects image data at a second compression ratethat is lower than the first compression rate.
 4. A multi-codec camerasystem comprising: a multi-codec camera that acquires video in the formof multiple compression-rate image data; and a server that is connectedto the multi-codec camera via a network and has a determination modulethat selects image data from the multiple compression-rate image dataacquired by the multi-codec camera according to a predeterminedreference, wherein the determination module of the server selects imagedata at a compression rate according to the condition of the networkload from the multiple compression-rate image data acquired by themulti-codec camera.
 5. The multi-codec camera system according to claim4, wherein when the condition of the network load is higher than apredetermined reference value, the determination module selects imagedata at a first compression rate, while when the condition of thenetwork load is lower than the predetermined reference value, thedetermination module selects image data at a second compression ratethat is lower than the first compression rate.
 6. The multi-codec camerasystem according to claim 4, wherein a client is connected to thenetwork, and when the condition of the server load is higher than apredetermined reference value, the server sends the client aninstruction to acquire image data directly from the multi-codec cameravia the network.
 7. A multi-codec camera system comprising: amulti-codec camera that acquires video in the form of multiplecompression-rate image data; a server that is connected to themulti-codec camera via a network and has a determination module thatselects image data from the multiple compression-rate image dataacquired by the multi-codec camera according to a predeterminedreference; and a client connected to the network, wherein the serversaves the multiple compression-rate image data acquired by themulti-codec camera, and the client uses image data at a firstcompression rate when the client searches the image data saved in theserver for image data that the client wants to acquire, while the clientacquires image data at a second compression rate that is higher than thefirst compression rate, when the client acquires the searched image datafrom the server.
 8. An image acquisition program executed on amulti-codec camera system including: a multi-codec camera that acquiresvideo in the form of multiple compression-rate image data; and adetermination module that selects image data from the multiplecompression-rate image data acquired by the multi-codec camera accordingto a predetermined reference, as program processes in the determinationmodule, the image acquisition program comprising the steps of: detectinga predetermined object from the video acquired by the multi-codeccamera; and selecting image data at a compression rate according to thesize of the detected object.
 9. An image acquisition program executed ona multi-codec camera system including: a multi-codec camera thatacquires video in the form of multiple compression-rate image data; anda server that is connected to the multi-codec camera via a network andhas a determination module that selects image data from the multiplecompression-rate image data acquired by the multi-codec camera accordingto a predetermined reference, as a program process in the determinationmodule of the server, the image acquisition program comprising the stepof: selecting image data at a compression rate according to thecondition of the network load from the multiple compression-rate imagedata acquired by the multi-codec camera.
 10. An image acquisitionprogram executed on a multi-codec camera system including: a multi-codeccamera that acquires video in the form of multiple compression-rateimage data; a server that is connected to the multi-codec camera via anetwork and has a determination module that selects image data from themultiple compression-rate image data acquired by the multi-codec cameraaccording to a predetermined reference; and a client connected to thenetwork, the image acquisition program comprising the steps of: as aprogram process in the server, saving the multiple compression-rateimage data acquired by the multi-codec camera; and as a program processin the client, using image data at a first compression rate when theclient searches the image data saved in the server for image data thatthe client wants to acquire, while acquiring image data at a secondcompression rate that is lower than the first compression rate, when theclient acquires the searched image data from the server.